Skip to content

#1210 Fixed highlighting of non-unicode chars in Output #1375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 31, 2022
Merged

#1210 Fixed highlighting of non-unicode chars in Output #1375

merged 1 commit into from
Aug 31, 2022

Conversation

kittaakos
Copy link
Contributor

Motivation

To avoid showing a warning in the Output if an unicode character is detected as

However, I could not develop a sketch that could reproduce the original issues reported in #1210.

unsigned long previousMillis;
void setup() {
  Serial.begin(9600);
}
void loop() {
  if (Serial.available() > 0) {
    while (Serial.available() > 0) {
      Serial.write(Serial.read());
      delay(10);
    }
  } else if (millis() - previousMillis >= 1000) {
    previousMillis = millis();
    Serial.println(u8"\U0001F60E"); // Unicode Character 'SMILING FACE WITH SUNGLASSES' (U+1F60E) https://www.fileformat.info/info/unicode/char/1f60e/index.htm
    Serial.println(u8"\u03BC"); // Unicode Character 'GREEK SMALL LETTER MU' (U+03BC) https://www.fileformat.info/info/unicode/char/03bc/index.htm
    Serial.println(u8"\u00A0"); // Unicode Character 'NO-BREAK SPACE' (U+00A0) https://www.fileformat.info/info/unicode/char/00a0/index.htm
    Serial.println(u8"\u2013"); // Unicode Character 'EN DASH' (U+2013) https://www.fileformat.info/info/unicode/char/2013/index.htm
    Serial.println(u8"\uFF09"); // Unicode Character 'FULLWIDTH RIGHT PARENTHESIS' (U+FF09) https://www.fileformat.info/info/unicode/char/ff09/index.htm

    Serial.write(0xEF);
    Serial.write(0xBC);
    Serial.write(0x89);
    Serial.println();
  }
}

If I modify the Output update code and forcefully print the following invalid characters (– ）), I can see the problem:
Screen Shot 2022-08-31 at 12 52 33

With the proposed changes, the problem goes away:
Screen Shot 2022-08-31 at 12 54 53

Change description

Other information

Closes #1210

Reviewer checklist

  • PR addresses a single concern.
  • The PR has no duplicates (please search among the Pull Requests before creating one)
  • PR title and description are properly filled.
  • Docs have been added / updated (for bug fixes / features)

@kittaakos kittaakos added topic: code Related to content of the project itself type: imperfection Perceived defect in any part of project topic: theia Related to the Theia IDE framework labels Aug 31, 2022
@kittaakos kittaakos mentioned this pull request Aug 31, 2022
Closed
3 tasks
per1234
per1234 approved these changes Aug 31, 2022
View reviewed changes
Copy link
Contributor

@per1234 per1234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this fixes #1210

Thanks Akos!

@per1234
Copy link
Contributor

per1234 commented Aug 31, 2022

I could not develop a sketch that could reproduce the original issues

A successful compilation of any sketch will produce the original issue if Arduino CLI is using the zh locale. The reason is these problematic characters are present in the sketch memory usage report that is printed at the end of every successful compilation. The English language version looks like this:

Sketch uses 662 bytes (0%) of program storage space. Maximum is 253952 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 8183 bytes for local variables. Maximum is 8192 bytes.

Unfortunately, this doesn't seem possible to achieve via the Arduino IDE 2.x "Language" preference because that sets the locale key of arduino-cli.yaml to zh-cn (which is the code used by the VS Code language pack) instead of zh.

But this sketch can also be used to reproduce it completely independently from locale:

#error （
void setup() {}
void loop() {}

image

(screenshot is of the IDE version without the fix)

@kittaakos kittaakos merged commit 204d71b into main Aug 31, 2022
@kittaakos kittaakos deleted the #1210 branch August 31, 2022 19:16
@kittaakos kittaakos mentioned this pull request Feb 22, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@per1234 per1234 per1234 approved these changes

@davegarthsimpson davegarthsimpson davegarthsimpson approved these changes

Assignees
No one assigned
Labels
topic: code Related to content of the project itself topic: theia Related to the Theia IDE framework type: imperfection Perceived defect in any part of project
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

The output is treated as the source code for correction prompts
3 participants
@kittaakos @per1234 @davegarthsimpson